The grammar guidelines of a particular natural language, e.g., English, Spanish, etc., impose structure on the natural language. Grammar guidelines for a particular natural language facilitate communication and understanding of spoken, and especially written, words in the language.
It is important that written information in a natural language follow the grammar guidelines of the language. Conformance to grammar guidelines is especially important in formal documents, such as research papers, resumes, and legal documents. Not only is conformance to grammar guidelines important in formal documents, conformance is expected. A formal document that does not conform to the appropriate grammar guidelines may be discounted or ignored by the audience of the document. However, it can be difficult to learn and correctly apply the various grammar guidelines of a natural language. For example, English has grammar guidelines for capitalization, hyphenation, etc., that may be counterintuitive.
Grammar tools have been developed to aid authors in conforming documents to the grammar guidelines of natural languages. Such tools may give an author feedback on applying one or more grammar guidelines to the text of a document. A grammar tool may have, encoded into the tool, one or more grammar rules. Each grammar rule is typically designed to detect violations of grammar guidelines. For any given grammar rule, there are one or more “target grammar guidelines” whose violations the rule is intended to identify. The value of a grammar rule hinges on (a) how frequently the “violations” identified by the rule are actual violations of its target grammar guidelines, and (b) what percentage of violations of its target grammar guidelines the rule actually catches.
It is important that a grammar rule correctly identify violations of its target grammar guidelines within the text of a document. For example, a tool that includes a grammar rule that is overbroad will produce “false positives” that identify, as violations, text that is not actually a violation. The production of too many false positives may frustrate the author. Furthermore, a poorly written grammar rule may be costly to apply to a document, may be redundant with already-existing grammar rules, or may even be contradictory to (a) other grammar rules or (b) its own grammar guidelines. Therefore, it is important to thoroughly test the performance of grammar rules before the rules are used to provide feedback to authors.
One method of testing the performance of a particular grammar rule is to apply the rule to a document, and to have a human inspect the document to determine whether the feedback generated by the rule is appropriate, given the content of the document. However, it is costly to have humans inspect documents by hand. A more efficient method of testing the performance of grammar rules is needed.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.